Graphics Processing Units (GPUs) have grown in recent years to support general-purpose computing. Due to the economic amounts of high resolution and real-time 3D graphics, GPUs have evolved into multicore processors that are multithreaded, extremely parallel, have high bandwidth capacity, and have the enormous computational capacity. Parallel processing is done with GPUs, which are multi-core processors. GPUs may improve the sustainability of parallel processing in circuit simulation. Our proposed work aims to create GPU architecture for parallelizing heuristic algorithms that focus on the following aspects: efficient data processing interaction between GPU and CPU, efficient memory management, and efficient parallelism function. Our theoretical GPU-accelerated parallel frameworks can significantly increase the performance of heuristic approaches for solving large-scale combinatorial optimization problems. We evaluated the effect of tests with our proposed GPU architecture to parallelize several heuristic approaches like simulated annealing, hill climbing, and genetic algorithms for solving combinatorial optimization problems. It compared our experimental findings to CPU-based sequential approaches for efficiency analysis, and all of our results demonstrate that parallelizing combinatorial meta-heuristics with our GPU platform offers better performing strategies in a valid period.